The present invention is directed towards versatile video transformation device and adaptive image processing methodology thereof to digitally scan convert, that is, reformat TV raster scan video and particularly high definition (HD) and/or digital DTV video data and associated synchronizing signals, for the purpose of making present standard television sets compatible at low cost with the latest advancements in free HD whilst allowing a multitude of other DTV programs and ancillary data to fill to a greater extent the remaining channel allocations by FCC to TV broadcasters.
Present TV sets are not capable of using HDTV, but rather are limited to a fixed-format and are incompatible with different TV raster formats such as that provided by HDTV. While an amount of versatility (and energy-efficiency) has been provided by U.S. Pat. No. 4,361,785, the disclosure of which is incorporated herein, it is desirable to improve and expand it, particularly, with regard to standard TV sets.
The inexpensive utility of versatility for single frequency standard TV is especially desirable now because of the availability of HDTV and DTV.
In the past, when color TV broadcasts became available, existing monochrome xe2x80x9cblack and whitexe2x80x9d TV sets, at the time, could not display the added dimension of color. Viewers could, however, still see in monochrome sets new programs broadcast in color because the novel chrominace signals conveying the hue and saturation (color purity) attributes were rationalized and compressed into the then existing video bandwidth without substantial interaction with the luminance. While the present invention is not directed towards improving the TV set circuitry per se, in so far as DTV system eliminates many visual impairments eg. interline flicker, sync jitter, moire, raster line structure, multipath ghosts, chrominace distortions etc. the picture quality of such TV sets should be markedly improved. This will be accomplished while, most importantly, addressing the broadcaster""s desire to allocate, as much as possible, other programs and data communications within the channel assigned to it by the FCC.
In this regard, the FCC has allocated the traditional broadcasters a new channel for DTV which can include xe2x80x9cmulticastingxe2x80x9d several simultaneous programs with conditional access for a nominal fee that would offset the costs of new equipment needed to broadcast DTV and free HDTV and also provide better competition with costly cable TV and satellite TV.
Ironically, the effective expansion of TV channel bandwidth capacity wrought by removal of redundancy in video compression techniques for HDTV appears directed towards the of viewers"" desire to receive more channels due to the quality of programming, or lack thereof. The improved quality of higher resolution picturesxe2x80x94that are not really discernible at present distance between set and viewer, is desirable and possibly only equally important to the viewer than the ability to receive additional channels. Accordingly, it appears that a need exists to provide a higher quality picture as that now made available by HDTV while allowing the broadcaster to utilize to the greater extent, the remaining portion of the channel allocation for additional programming or xe2x80x9cdata castingxe2x80x9d.
As to the first objective, which, if addressed properly, allows the second objective to be met, the history of TV evolution reveals the perennial problems needing solution to meet the resolution revolution that now requires brighter, smarter picture processing to live up to HDTV promises of 35 mm quality.
Naturally, the periphery of the picture is blurred by TV cameras that pan, tilt and track to keep in the center of TV moving images that are center of attraction such as an ice skater or player with the football, basketball, puck etc. From empirical human factors data obtained from TV trackers and xe2x80x9cmoving windowxe2x80x9d scan converters (Rand Corp., RCA Corp., Fairchild Camera et al.) a target should move through the standard TV field of view in 4 seconds or more but definitely no faster than one second. One second equals time for refreshing 60 fields and also 3.4 fixations of 5xc3x975 degrees at 0.3 second assuming 480 scan lines each subtend arc minute defined as 20/20 vision.
Thus those conclusions that an object ought not to move more than 1-to-4 scan line in {fraction (1/60)}th second field is consistent with video compression ratios of 30-120 since 1.7%-0.42% or less is changed in either or both axes.
The prior art of monochrome video scan conversion was considered to be a linear system using analog and later digital means with predictable performance. One of the earliest scan converters involved a TV camera scanning a radar PPI plan position indictor whereby the video values at xe2x80x9crho-thetaxe2x80x9d (range and angle) polar coordinates driving an oscilloscope display with long persistence phosphor were transformed to the x,y Cartesian coordinates of the horizontal and vertical deflection in TV camera feeding a synchronized TV display. It and subsequent scan converters also used to convert diagnostic ultrasound and sonar signals to TV images mechanized the coordinate transformation matrix called the Jacobian in Calculus texts.
Generally, when higher resolution ultrasound or radar indicators or TV rasters eg. 1225 lines EIA-RS-343 are scan converted to standard 525 lines EIA-RS-170, 330 severe Moire patterns form by the beat-frequency between the horizontal line frequencies if the readout beam was not sufficiently defocussed to span several input scan lines. Preferably an anti-Moire digital filter or dither analogous to analog xe2x80x9cwobblexe2x80x9d signal can modulate the scan converter input PPI deflection as a function of rho, theta to widen the spokes as it were near the rim when nearly parallel to horizontal scan lines readout.
A patent in the early 1970""s granted to Don Dudley of Dataplex Inc. explained how 1225 line HDTV could be broadcast in place of redundant fields via standard 525 line TV. In principle, that is what HDTV does today in transmitting the differences between fields with more robust algorithms for removal of redundancy subject to acceptable rates of distortion and the empirical 70% Kell factor or actually 2/pi=64% modulation transfer function (MTF) at Nyquist sample limit.
Another early scan converter involved the simplistic reduction of 625 TV lines 25 frames/second in Europe to 525 TV lines 30 frames/second in USA by cropping off 100 lines, including at times the forehead of the televised person. Subsequently, instead of cropping 50 lines from the top and bottom, 4 of every 25 lines were dropped out or xe2x80x9cdecimatedxe2x80x9d since 625/525=25/21. Preferably, as DTV FSC demonstrated in 1985, for each and every one of 21 readout lines falling between 2 of 25 lines weighting factors proportional to their separations enabled the interpolation of video values between the 625 lines and smoother decimation to 525 than simplistic drop outs. Likewise for example, 720p*2/3=1080i*4/9=480.
In addition to spatial artifacts of aliasing from subsampling and of contouring from quantization in digital scan conversion, the different frame rates created temporal problems with moving images analogous to the stroboscopic effect of backward wagon wheels in sampling 25-30 frames/second. The modern solution by Silicon Graphics et al. is known as xe2x80x9cmorphingxe2x80x9d is disadvantageous since its temporal interpolation/decimation between fields would require brute-force RAM or excessive software xe2x80x9cbloatwarexe2x80x9d for MPEG-2 decoding.
Before digital random access memories (RAM) became inexpensive and commonplace, instead of a TV camera tube scanning one CRT picture and driving another CRT, Texas Instruments, Tektronix, Hughes Image Devices and other companies produced scan converter tubes with electronic input (write) and output (read) video whose associated horizontal and vertical deflection coils were independent in so called xe2x80x9cdouble-endedxe2x80x9d tubes simultaneous read and write. They were time-shared in cheaper xe2x80x9csingle-endedxe2x80x9d tubes as in RAM. Most of these CRT scan converters were eventually replaced by digital image processors or TV xe2x80x9cframe grabbersxe2x80x9d using RAM in a xe2x80x9cbrute-forcexe2x80x9d manner because of the Nyquist sampling limit since RAM was so inexpensive.
Just as the Taylor Series representation in the time domain of the video signal by summation of its time derivatives is the mathematical basis for interpolation or reading video values between the scan lines so also the Fourier series representation in the frequency domain by summation of harmonics of complex sinusoids is the basis of video compression in terms of DCT Discrete Cosine Transform that is the real part of the Fourier Transform. Accordingly, the video transformation may be mechanized in either the time domain as in all known scan converters or in the frequency domain as will be discussed.
While in the past various approaches addressed changes in broadcasting formats and changes in the industry, presently what has not been addressed successfully is the ability to provide for enhanced quality TV resolution as provided for by HDTV and DTV in conventional TV while reserving as much as possible or desired of the remainder of the channel allocation for other programming and/or ancillary data. This is something that is desired by both the viewers and the broadcasters.
It is an object of the present invention to provide the fullest fidelity, video data integrity of HDTV within the constraints of standard TV sets which is cost effective.
It is a further object to provide externally image quality enhancements eg. xe2x80x9csparklexe2x80x9d of broadcast formats for HDTV so as to optimize the flicker-free field of view or sparkling clarity of TV and provide for full view of the uncropped HDTV picture or full use of-present TV screen.
It is also an object to provide buffer storage for refreshing of compressed images and for video transformation without having to decode and again compress and re-encode the HDTV. In this regard, briefly, the present invention provides scan conversion in the frequency domain using the equivalent Fast Fourier Transforms (FFT) of time scaling, shifting and of video derivatives for interpolation/decimation because N nodes are efficiently reduced from Nxc3x97N operations to Nxc3x97Ln(N) and because it exploits benefit of compression ratio which is proportional to resolution.
It is a further object to provide without objectionable aliasing or flicker remarkably higher resolution than is ordinarily expected by Nyquist sampling theorem and averaging circuits. In this regard, briefly, the present invention effectively circumvents the luminance blurring and chrominance cross talk at edges due to averaging or due to low pass filters at half the sampling frequency and dithers spatial filter sparkle to smooth flicker.
It is also an object to provide video transformation of DTV noninterlaced, so called progressive scan to standard interlaced raster format and conversely transform interlaced fields to noninterlaced frames for PC monitors and printers.
It is a further object to provide enabling technology and local storage for compact video router transformations to allow and to promote individually addressed video transmissions through quasi cross-point switches connecting any number of video sources to any number of video destinations.
It is a yet further objective of versatility in expandability up to 65Kxc3x9765K pixels of MPEG so as to provide the sensation of realism.
Lastly, it is the ultimate object of the invention to provide for enhance quality TV resolution as that provided for by HDTV while reserving the remainder of the channel allocation for additional programs and ancillary computer data.
The present invention provides for the foregoing objectives by providing for a means to receive, decode and scan convert 1920xc3x971080i or 1280xc3x97720p HDTV or other DTV format on standard TV sets while substantially enhancing display thereof. It provides for sampling and filtering of the higher definition TV images for scan conversion without typical crosscoupling of chrominace and luminance so as to sparkle on lower resolution TV sets while substantially maintaining the input detail subject to viewers complex spatial/temporal flicker tolerance. It does this by a dithering of raster origin offsets in zigzag fashion within 3xc3x973 or 2xc3x972 array of pixels for scan conversion into one pixel readout. This allows for an HDTV quality picture to be broadcasted on a format having a reduced bandwidth so as to allow the broadcaster to have a greater portion of the remaining channel for other programming whilst allowing a standard TV to utilize the HDTV or DTV format broadcasted.
The present invention also provides for renewable conditional access to demultiplex multicast programs and ancillary data compressed in packets within one DTV channel allocation. It enables standard TV sets to become compatible with DTV, HDTV and flexible formats of PC monitors and hard copy printers.
Moreover, its electronics may be incorporated within newer TV sets to minimize redundancy of hardware and its electronics may be incorporated within a integrated circuit chip.